Speech recognition capability for a personal digital assistant

ABSTRACT

A speech recognition module for a personal digital assistant comprises: a module housing designed to engage with an accessory feature of the PDA, such as an accessory slot; a microphone for receiving speech commands from the person; and a speech recognition system. A corresponding electrical speech command signal is communicated to the portable computing device, allowing control of the operation of a software application program running on the portable computing device. In particular, menu items can be selected for generation of a diet log for the person, for example during a weight control program.

FIELD OF THE INVENTION

The invention relates to speech recognition capabilities of a computingdevice. In particular, the invention relates to the use of speechrecognition in creating a diet log for a person on a weight controlprogram.

BACKGROUND OF THE INVENTION

Portable computing devices such as personal digital assistants (PDAs),including Palm Pilots, Handspring Visors, hand-held PCs, and similardevices, have become incredibly popular because of their functionalityand flexibility. For purposes of this application, the term computingdevice should be understood to include all computing devices, includingthose with additional functionality, including: desk-top, laptop,hand-held, palm top and wearable computers; and also cellular phones,telephones, two-way paging devices, radios, communicators, calculators,organizers, e-books, navigation devices, automobile electronics, digitalinteractive television controllers, and other electronic devices havingcomputing capability. However, for ease of description, this applicationwill focus on portable computing devices such as the Palm Pilot familyof hand-held devices (Palm Inc., Santa Clara, Calif.) and the HandspringVisor (Handspring, Mountain View, Calif.).

These personal digital assistants (or PDAs) include buttons and touchsensitive screens allowing input of data in a variety of ways, includingvarious forms of handwriting recognition and simplified keyboards. Forsome persons and applications, these forms of data input areinsufficient. For example, a person who wishes to input information,such as recording what meals were eaten during a day or expensesincurred on a trip, must grasp an input stylus with one hand whileholding the PDA in the other hand. The stylus is then used to touch thetouch sensitive screen in a manner indicative of the input desired. Thisobviously ties up both of the person's hands, and requires a moderatelevel of visual and mental focus. It would be more desirable if datainput could be made using a person's voice, rather than manualmanipulation.

A partial solution is offered by some hand-helds which include voicerecording capability. A person who wishes to quickly record informationon the PDA presses a record button and speaks into a microphone. Later,the recorded information may be played back. Unfortunately, thisapproach falls far short of what is desirable. The voice recording isonly voice-to-voice and therefore someone must later listen to therecorded voice and manually input the data. While the original recordingstep may be easier than manual manipulation, manual manipulation is notavoided because it must be completed at a later time. More desirablewould be a voice-to-text voice recognition capability so that a personmay speak commands and information and the PDA responds to the commandsand records the information in text form. However, voice-to-text speechrecognition is extremely complex and requires powerful computerprocessors and a large data storage capability. Current PDAs offer onlya fraction of the necessary processing power and storage.

Speech recognition software and systems are well known in the art, forexample as described in U.S. Pat. Nos. 5,749,072 to Mazurkiewicz et al.,and 6,167,255 to Kennedy III et al, incorporated herein by reference.

SUMMARY OF THE INVENTION

According to the present invention, the limitations of currentlyavailable computing devices (such as portable computing devicesincluding PDAs) and speech recognition software are overcome by offeringa simplified speech recognition program and software designed to workwith this simplified speech recognition capability. Much of thecomplexity of speech recognition software comes from the huge variety ofwords in a given language and the similarity in sounds of many of thesewords. The software must be capable of very accurately capturing thespoken sounds. Then, the software must determine what word was spokenout of a variety of similar choices by analyzing the context of theword, statistical likelihood, and past usage by a given person. Also,persons do not consistently pronounce words so software must be“trained” to recognize a specific person. The speech recognitionchallenge is significantly reduced by artificially constraining thevocabulary the software and hardware must interpret. Examples ofrestricted vocabulary speech recognition are currently used on manytelephone systems, allowing persons to either press a number 0 through 9or to speak the number. In this situation, the speech recognitionsoftware must only recognize and distinguish between ten choices, all ofwhich are reasonably distinct.

According to the present invention, a similar capability is provided toPDAs. In a preferred embodiment, the PDA is capable of recognizingnumbers 0 through 9 when they are spoken by a person. Current PDAs donot have this hardware or software capability. However, software may beadded to most PDAs. Also, most PDAs are capable of accepting add-onhardware modules. For example, the Handspring Visor has a slotarrangement on its back side known as a Springboard. A variety ofhardware modules, such as memory modules, may be inserted into theSpringboard to enhance the capability of the Handspring Visor. Likewise,the Palm Pilot family of hand-held PDAs accept clip-on hardware modulesto enhance their capabilities. It is expected that future versions ofPDAs will include new and simplified hardware attachment capabilities.

Typically, a hardware module which is inserted into or attached to a PDAmay also include application software further enhancing the capabilitiesof the PDA and taking advantage of the additional hardware. For example,the Handspring Visor accepts numerous Springboard compatible hardwaremodules, such as digital camera modules and music recording and playbackmodules. These modules interconnect with the Springboard portion of theHandspring Visor and include hardware such as a digital camera lens andrelated electronics, or music recording and playback hardware. Thisadd-on hardware cooperates with the existing capabilities of theHandspring Visor to provide the capabilities of a digital camera ordigital music player and recorder. The hardware modules also includeonboard software which is accessed and used by the hardware of theHandspring Visor and the add-on hardware module. The software allows themodule and Handspring Visor to communicate and provide the enhancedcapabilities. Likewise, hardware modules such as wireless modems may beinterconnected with some members of the Palm Pilot family.

A speech recognition module for a personal digital assistant comprises:a module housing configured to engage with an accessory feature of thePDA; a speech receiving device for receiving speech commands from theperson; and a speech recognition means capable of distinguishingmultiple speech commands and communicating a corresponding set ofelectrical commands to the PDA. Further, the speech recognition modulefor a portable computing device can comprise: a module housingconfigured to engage with an accessory interface of the portablecomputing device; a speech receiving device, adapted to receive a speechcommand from the person and to provide an electrical speech signal; anda speech recognition system, adapted to receive the electrical speechsignal from the speech receiving device, to distinguish the speechcommand, and to communicate a corresponding electrical speech commandsignal to the portable computing device, wherein the electrical speechcommand signal is used to control the operation of a softwareapplication program running on the portable computing device. The speechreceiving device can be a microphone. Speech recognition of microphonesignals is disclosed in U.S. Pat. No. 5,749,072 to Mazurkiewicz et al.,for example with reference to FIG. 3 of that patent. The module canfurther comprise a nutritional database of food item identifiers andassociated nutritional data for a number of food items. Nutritional datacan comprise calorie content, fat content, fiber content, proteincontent, glycemic index, mineral content, vitamin content, and othernutritional components associated with each of the food itemidentifiers. The software application program can be adapted to select aselected food item identifier from the nutritional database undercontrol of the electrical speech command signal, and to store theselected food item identifier and associated nutritional data in amemory so as to create a diet log for the person.

A portable computing system according to the present inventioncomprises: a display; a microprocessor; a memory; a database ofnutritional data relating to each of a plurality of food itemidentifiers; a speech recognition system providing an electrical speechcommand signal correlated with a spoken input; and a softwareapplication program, executed by the microprocessor, adapted to displaya food menu of food item identifiers on the display, to select a fooditem identifier from the food menu based on the electrical speechcommand signal received from the speech recognition system, and to storenutritional data relating to the selected food item identifier in thememory so as to create a diet log of selected food items. Thenutritional database can be arranged in a hierarchical form in thatparticular food items are characterized by a generic type, and in thesoftware application program can be further adapted to display a menu ofgeneric types on the display, and to select a generic type from the menuof generic types on receipt of the electrical speech signal.

An improved accessory module for a portable computing device (thecombined system which can then be used for diet logging as part of aweight control program) comprises: an electrical interface, adapted tocommunicate with an accessory port of the portable computing device; amemory; a nutritional database, stored in the memory, correlating eachof a number of food item identifiers with an associated food itemnutritional content; and a software application program, stored in thememory, and executable by the microprocessor so as to present a menu offood item identifiers on the display, further adapted to select a fooditem selection from the menu of food item identifiers on receipt of aspeech command signal, and further adapted to store the food itemselection and correlated nutritional content within a diet log for theperson. Nutritional content may included calorie content, diet componentcontent, and other nutritional component of the food item. The speechcommand signal can be provided by a speech recognition system residenton the accessory module, on the portable computing device, or on anotherdevice in communication with the portable computing device.

A method of creating a diet log for a person, comprises: presenting amenu of food type identifiers to the person; receiving a type selectionspeech command from the person; correlating the type selection speechcommand with a selected food type identifier; presenting a menu of fooditem identifiers to the person, wherein the food item identifiers have ahierarchical relationship to the selected food type identifier;receiving an item selection speech command from the person; correlatingthe item selection speech command with a selected food item identifier;and storing the selected food item identifier and associated nutritionaldata within a memory device, so as to create a diet log for the person.The selected food item identifier is correlated with associatednutritional data using a nutritional database. The menus can bepresented to the person on the display of a portable electronic devicecarried by the person, such as a personal digital assistant (PDA), otherportable computing device, wrist-mounted device, wearable computersystem, electronic book, electronic notebook, digital organizer,wireless phone, desktop computer system, web-TV, digital interactive TV,tablet computer, visor-mounted display system, or the like. The menuspresented to the person can be chosen based on the physical location ofthe person, for example as input by the person, as transmitted from alocal wireless network, or as determined by a position location systemsuch as a global positioning system (GPS). The correlation of speechcommands with selected identifiers can performed by a softwareapplication program ruining on the portable computing device, accessorymodule to the portable computing device, or other device incommunication with the portable computing device such as a remotecomputer system in communication with the portable computing device overa communications network.

In other embodiments, spoken commands can be recorded in a memory andanalyzed later.

Computing devices may further include systems such as a separate speechreceiving device (for example, a wrist mounted device), processor module(for example a belt mounted computer comprising a processor, memory, andplug-in hardware module comprising a speech recognition system), anddisplay device (for example, a visor-mounted display). Preferablycomponents are in wireless communication and function as a unitarydevice.

Wireless transmission methods can include Bluetooth, a wireless protocoldeveloped by Ericsson (Sweden) and others, IEEE802.11(b) and similar,local wireless networks such as wireless Ethernet, IR, optical, andultrasound methods. Wireless transmission methods between devices offersimproved convenience, however cable connections can be used in place ofwireless methods.

U.S. provisional application Serial No. 60/212,319 to James R. Mault(filed Jun. 16, 2000) is incorporated herein by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a hardware module being inserted into a portable computingdevice;

FIG. 2 shows a portable computing device having an inserted hardwaremodule;

FIG. 3 shows a portable computing device having an inserted hardwaremodule;

FIGS. 4-6 show possible menu presentations on the display of a device,by which a person can select a cereal name to add to a diet log;

FIGS. 7-9 show other possible menu presentations on the display of adevice, by which a person can select a cereal name to add to a diet log;

FIG. 10A shows a simplified schematic of a portable computing devicewhich may be used in embodiments of the present invention;

FIG. 10B shows a simplified schematic of a hardware module which may beused in embodiments of the present invention;

FIG. 11 is a flowchart illustrating a method of diet logging;

FIG. 12A shows a presentation on a display, whereby a recognized letteris used to restrict menu presentation;

FIG. 12B shows a further presentation on a display, whereby a recognizedletter is used to restrict menu presentation; and

FIG. 13 shown a schematic of a system in which the nutritional databaseand speech recognition system are accessed using a communications linkto a remote server.

DETAILED DESCRIPTION OF THE INVENTION

According to the present invention, a hardware module is provided whichinterconnects with a portable computing device, such as a PDA. Thehardware module includes a microphone for receiving speech and mayinclude other controls such as an on-off switch and sensitivityadjustments. An example of the present invention is shown schematicallyin FIGS. 1 and 2. The portable computing device 10 receives a hardwaremodule 12 which, in the Figures, slides into a slot 16 in the back sideof the portable computing device 10. The hardware module 12 includes amicrophone 14 located near the edge of the module so that it is on theupper side of the portable computing device in use. In portablecomputing devices already including a microphone, the microphone may beomitted from the module. The module preferably includes both hardwareand software so that when it is inserted in the slot in the portablecomputing device, the portable computing device becomes capable ofsimplified speech recognition.

FIG. 3 shows another embodiment, in which module 26 is inserted into aslot in the lower edge of a portable computing device 20. Part of themodule is exposed, allowing a person to speak into microphone 28. Theportable computing device has display 22 and data entry keys 24. Thedisplay is divided into a menu display area 22 a and a user instructionarea 22 b. However, this division is not critical to embodiments of thepresent invention.

Referring now to FIGS. 4-6, an example use for the simplified speechrecognition module will be explained. In the Figures, a portablecomputing device is shown displaying textual menus for providing anumber of choices. The hardware module is not shown. In this example,the portable computing device is being used to record dietary intake. InFIG. 4, the portable computing device presents a menu with six optionsfor meal type. Software providing the dietary intake software may bepart of the hardware module, an additional module, or may be softwareloaded into, or already part of, the portable computing device. Usingthe speech recognition module, the person merely speaks the number oftheir choice of meal type. For example, if the person wishes to inputinformation related to breakfast, they say the word “one.” This isequivalent to entering the number 1 using the stylus or button on theportable computing device. This choice brings up a second menu, as shownin FIG. 5. In this menu, several breakfast choices are displayed. If theperson wishes to choose hot cereal, they pronounce the word “two.” This,once again, enters the choice into the software causing the software tobring up a third menu, as shown in FIG. 6. This third menu presents avariety of choices for hot cereal. It will be clear to those of skill inthe art how this process may continue, in a very simplified manner,allowing a person to choose items for their diet. The chosen items, andassociated nutritional data, can then be stored in a diet log for theperson. This method may be similarly applied to data entry chores suchas expense account tracking.

As known to those of skill in the art, a very wide variety of softwareapplications are available for portable computing devices. According tothe present invention, the hardware module and the software it carriesare designed to work with many of the software applications. That is,the hardware and software module integrates with and communicates with aportable computing device such that speaking a number provides the sameinput to the software running on the portable computing device as if theperson had used a stylus to input the number. Also, additionalvocabulary may be provided in the same hardware module, or anotherhardware module according to the present invention. For example, manyportable computing device functions require the use of up and downbuttons. For this purpose, the hardware module may also recognize thecommands “up” and “down.”. Commands could also be recognized for a fewof the most commonly used functions of the portable computing device,such as “date” for date book, “to do” for the to do list, and “address”or “phone” for the address list, and to change operating modes of theportable computing device. The hardware module may also recognize a fewletters such as A through J, or the entire alphabet, a set ofalphanumeric characters, or a set of words corresponding to characters,for example alpha, bravo etc. for the alphabet A-Z. In addition, a smallnumber of extra words such as back, delete, enter, and the like, forsimple navigation can be added to the limited list recognized.Obviously, the speech recognition capabilities are entirely dependent onthe complexity of the hardware and software in the module. Therefore,for some applications a very simple speech recognition module may besufficient. In this case, recognizing numerals 0 through 9 may be morethan sufficient. As the software and hardware capabilities of portablecomputing devices improve, more complex modules may be desirable forsome applications.

The hardware module can be a proprietary accessory to a portablecomputing device, or may comply with well known standards, for examplePCMCIA and Miniature Card standards. The design of an accessory modulewith imaging capabilities, which may be advantageously modified for usein the present invention by inclusion of a voice recognition system, isdisclosed in U.S. Pat. No. 6,219,560 to Erkkila et al., incorporatedherein by reference.

As will be clear to those of skill in the art, the present inventionprovides numerous advantages in the input of data. In the example givenearlier in entering diet information, the person may simply enter theirchoices by quickly repeating the words “one,” “two,” “one.” This may beboth easier and more pleasant than manual input for many persons. Also,some persons may have limited dexterity, making use of a stylus,keyboard, or other manual entry mechanism very difficult. The speechrecognition module allows single handed, or even hands free, use sincethe stylus need not be grasped.

In other embodiments, a voice reception module comprising a microphoneand a transmitter can be mounted near the person's mouth, such as on acollar, or the skin, and transmit audio signals to a computing device,such as a PDA, desktop computer system, remote server, interactivetelevision set top box, and the like. The person can view a menupresented on the display of a computing device, and select items usingspeech commands directed to the microphone. The voice reception modulecan further comprise an oscillator, such that the signal received by themicrophone is modified by the shape of the person's mouth, for exampleby the action of forming words (such as letters, numbers, food names) sothat, for example, the person can provide commands to the computingdevice by silently forming words without the need to speak audibly.

Diet Log

As described above in relation to FIGS. 4-6, speech control of menuselections is useful in creating a diet log. In U.S. Pat. Nos. 5,704,350and 4,891,756, incorporated herein by reference, Williams describes theselection of food items through a hierarchical menu system. Thishierarchical approach can be advantageously combined with speechrecognition for diet log creation. A diet log software applicationprogram was also described in U.S. Provisional Pat. App. No. 60/240,185,filed Oct. 13, 2000, incorporated herein by reference. This program canbe advantageously modified for use with embodiments of the presentinvention by adapting the menu selection methods provided by thesoftware program to be responsive to spoken commands, as interpreted bya speech recognition system. Other diet log software are known in theart, for example as described by Kretsch in U.S. Pat. No. 5,233,520.

In embodiments of the present invention, the person is initiallypresented with a high-level menu of food types, such as a menu listingas shown in FIG. 7. As described above, the person can speak the numberassociated with each food type identifier. The person may also speak aletter corresponding to the first letter of one or more of the menulistings. The menu listing can be reduced to a unique item, which isthen selected, or to a number of items each beginning with the spokenletter. The person can then speak a second letter to further restrictthe selection, and a third letter if necessary.

If multiple word listings are presented to the person, a second (orlater) spoken letter can be applied to restrict choices to those havinga second word beginning with that letter. For example, if breads arelisted in a menu as bread, rye; bread, brown; bread, pumpernickel (andthe like), a person may speak “B . . . R” to select bread, rye.

With respect to FIG. 7, a person may speak “1”, or “C” to choosecereals. In other embodiments, the vocabulary of the speech recognitionsystem is automatically restricted to the words and numbers displayed onthe menu. In this case, the word “cereals” can be reliably distinguishedfrom the other word options by a speech recognition system. In thiscase, the first syllable alone can be used to distinguish the word“cereals” from the other choices.

FIG. 8 shows another menu of food item identifiers which may bepresented to the person after the person has chosen cereals from themenu of FIG. 7.

FIG. 9 shows a portion size menu which may be presented to the personafter they have selected a food item identifier.

FIG. 10A shows a schematic of a portable computing device which may beused in embodiments of the present invention, comprising microprocessor(processor) 32, memory 34 (which may comprise ROM, RAM, memory modules,and other memory types and combinations), user interface 36 (which maycomprise buttons, stylus, and other mechanical data entry mechanisms), adisplay 38 (comprising a display driver), peripherals 39 (which maycomprise an additional display, indicator, printer, buzzer, speaker,modem, drive, and the like), accessory interface 42, and hardware module44. FIG. 10B shows a possible schematic of a hardware module 44 for usein embodiments of the present invention comprising a microphone 46,analog to digital converter 48, signal processor 50, speech analysiscircuit 52, and nutritional database 58. In this example, commands suchas numeric speech are interpreted by the analysis circuit 52, whichsends a corresponding electrical signal to the interface 42, forinterpretation by the microprocessor 32. The nutritional database canalso be accessed by a software application program running on theportable computing device. In other embodiments, the module 44 mayfurther comprise a processor and additional memory, and diet logsoftware may run on the module, displaying information on the display38, and storing a diet log either on the memory of the module, in thememory 34, or at a remote location such as a remote computer accessedover a communications network.

FIG. 11 shows a method of creating part of a diet log according to thepresent invention. Box 70 corresponds to the selection of the operatingmode of the computing device, for example by speaking the word “diet”.Other words which may be recognized at this stage might include exercise(for exercise logging), balance (for calorie balance calculation),address (for address book), and the like. Box 72 corresponds to thepresentation of a diet log welcome screen to the person, possiblyincluding a summary of data entry methods. Box 74 corresponds to the useof available information to modify presentation of menus to the person.Box 76 corresponds to the presentation of types of food to the person,for example as appropriate for the time of day, or other informationdetermined in step 74. Box 78 corresponds to the selection of a type offoods, for example using voice recognition methods. Box 80 correspondsto the display of food items according to the selected food type. Box 82corresponds to the selection of a food item by identifier, for exampleby a particular name or code. Box 84 corresponds to the display of amenu of related information. Box 86 corresponds to the selection ofrelated information.

With regard to Box 74, available information can include location,purchase information, time, known preferences, known meals supplied by aweight control business, dietary goals, and the like. The informationcan be in the memory of the computing device, received over acommunication network, preprogrammed, based on previous behavior, ordetermined using other methods and sources. For example, at timescorresponding to lunch times, a lunch menu can be presented. Other mealand snack times can be correlated with the consumption of certain foods,possibly using previously collected data. If the location of the personis known (for example using global positioning, cell phonetriangulation, or local wireless network based methods) the menu can bepresented accordingly. A person in a particular restaurant can bepresented with diet log options corresponding to that restaurant. Aperson at home can be presented with lists of food known to be at home,for example using prior purchase information or known preferences. Ifthe person as at a business office, typical foods consumed there can bepresented as a menu. There is often a strong correlation between aperson's location and the foods that they consume. This correlation canbe established by a software learning process, or can be preprogrammed,or entered by the person. The food-location correlations can then beused to restrict menu presentation to the person. If food has beenobtained from a vending machine or similar food dispensing machine, orsuch a machine is detected nearby, menus corresponding to availablefoods can be presented. Personal expenditure information can also beused, for example if a transaction has taken place at a restaurant orfood retail establishment, that information can be used to helpdetermine food consumed. If the identity of a previously purchased mealis known, for example using a credit card transaction log, the identityof the food can be entered into the diet log and the methods of thepresent invention can then be used to record the amount of food eaten,for example as a fraction of serving size.

With further regard to Box 86, further information can comprise portionsize, preparation method, accompanying items or omissions, fraction ofmeal eaten, and the like. For example, if the person chooses cornflakesas the food item identifier, the person can then be asked to enterportion size and amount of milk. This can be done by presenting a menuof options and receiving spoken choices, possibly using more than onemenu depending on the food item selected in box 82.

FIG. 12A shows a display (100) of a computing device having a window 102displaying received speech commands. In this example, the letters A-Zare used for speech commands, but other limited vocabularies such asnumbers can also be used. The person has spoken the letter “C”, which isrecognized by the speech recognition system and displayed at location104 in window 102. A menu of food item identifiers (or types, accordingto the diet log method and hierarchy in use) beginning with the letter“C” is displayed. If the person has just eaten cabbage, the person thenspeaks the letter “A” to distinguish it from the other displayed option.Further letters can be entered to further restrict the menu ifnecessary, with an item being selected once it is the only menu optionremaining. The word “Back” (or similar) can be recognized to delete adisplayed letter and remove the corresponding restricted selection offood item identifiers.

FIG. 12B shows the letter “K” displayed in window 102, and two diet logoptions presented, kumquats and pickled kumquats. In this case, thepickled option can be selected by speaking the letter “P”, the firstletter of the second word. This general approach is valuable in dietlogging, as often a single food item name will be repeated with severalmodifiers, or plural modifiers. In this case, speaking the first letterof the base name (e.g. “M” for milk), and the first letter of themodifying term(s) (e.g. “S” for skim), can be used for rapidspeech-recognition selection of diet log items. The first letter of thebase name can be repeated to select the base (unmodified) items. Otherletters or numbers can be entered to differentiate ambiguities ifnecessary. Alternatively, the modifiers can be numbered and the personspeaks a number corresponding to the modifier. In more advanced systems,the recognized vocabulary of the speech recognition system is restrictedto the options presented to the person, which considerably increases thereliability of the speech recognition system over one trying to matchspoken commands to a large available vocabulary. In some embodiments,only the first spoken syllable, or first two syllables, are used forspeech recognition.

Nutritional Database

The hardware module described above can comprise a memory containing anutritional database. This database comprises food item identifiers,such as the names of foods, meals, beverages, diet supplements, and thelike. The data can be arranged in a hierarchical fashion, so that fooditem identifiers are grouped by generic type, such as: meat products,dairy products, starches, preferred meals, lunch items, breakfast items,dinner items, meal supplements, nutraceuticals, prepackaged foods, andthe like. The generic types can further be grouped into higher levelgroups, or divided into sub-groups, if convenient. Presented menus canin certain cases comprise both food types and specific food identifiers.

A nutrition database can also be contained in the memory of a portablecomputing device, or on a memory module which may be inserted into theportable computing device or the hardware module described above, usinga memory module slot.

The nutritional database can also be located on another computer system,memory module, or device in communication with the portable computingdevice, as will be described in more detail below.

Food items can be added to the nutritional database using the availabledata entry mechanisms of the portable computing device, or using datareceived over a communications network or memory module transfer.

Communications Network

The portable computing device can be in communication with a remotecomputer system over a communications network such as the Internet. Forexample, the portable computing device can be in wireless communicationwith the Internet through an Internet service provider (ISP), and thenthrough the Internet to a remote server system. (The ISP is consideredpart of the communications network). In this case, the nutritionaldatabase and speech recognition system can be located on the remotecomputer system.

In other embodiments, a voice signal, for example as provided by amicrophone or other speech receiving device, can be transmitted over thecommunications network to a remote computer system, which can comprise aspeech recognition system. A diet log software application can beexecuted by the portable computing device or remote computer system.Menu displays, nutritional information, diet advice, and the like can bepresented on the display of the portable computing device. The necessarybandwidth for uploading voice signals will likely be much less than thatnecessary for downloading visual data from the remote computer system tothe portable computing device. Hence, different communications networkscan be used for data transfer to and from the portable computing device.For example, a wireless phone system can be used to transmit voice datato the remote computer, and a higher bandwidth wireless connection to anInternet service provider can be used to receive data for presentationon the display.

FIG. 13 shows a portable computing device 120 in communication with aremote server system 124 over communications network 122, which can bethe Internet. Spoken commands received by the device 120 are transmittedas digitized audio files to the server 124, which directs them to thespeech recognition system 126. The spoken commands are then convertedinto computer-readable commands (electronic commands, or speech commandsignals), and a server software application program can responds to theelectronic commands by selecting data from the nutritional database.Menus generated by the server software application program can be viewedon the display of the computing device 120.

Product Codes

As part of a weight control program, a weight control business cansupply a person with prepackaged meals. These can be identified usingabbreviated codes for convenient entry into a diet log. For example, alunch can be identified by a code such as L1, which can be readilyentered into a diet log by speech command or other method. Thenutritional database can further be supplied by the weight controlbusiness, so as to contain nutritional information correlated with eachproduct code.

Product identities and/or nutritional content can be algorithmicallyconverted to a numeric (or alphanumeric) code, analogous to Gemstar VCRPlus codes. This facilitates the entering of this simplified data into adiet log by speaking the code characters.

Exercise Log

The person can also record exercises on the portable computing device inan analogous manner. Exercises and activities can be grouped in ahierarchical fashion within an exercise database, contained within thememory of the portable computing device, a module, or other device incommunication with the portable computing device. The portable computingdevice can have several operating modes, such as diet log, exercise log,calorie balance calculator, time display, position display, organizer,word processor, and the like, and can be switched between modes by anyconvenient method.

Calorie Balance

The person can determine their resting metabolic rate (RMR) using ametabolic rate meter such as an indirect calorimeter. The RMR can beentered into the portable computing device by any convenient method,such as be speaking the digits combined with speech recognition. Thecalorie management software can then be used to calculate caloriebalance using diet log entries, exercise log entries (and/or estimatedactivity levels), and RMR.

PDA-Cell Phone

As is known in the art, a portable computing device such as a PDA can bereadily combined with the functionality of a wireless telephone into aunitary device, which we will term a DAP (digital assistant—phone). Inthis case, a microphone will be present in the DAP and can be used forspeech command operation of a diet log software. The speaker of the DAPcan be used to provide feedback to the person, such as noises toindicate successful or unsuccessful speech recognitions. The termportable computing device, as used herein, includes devices withadditional functionality such as a DAP.

Using a portable computing device with a speaker, speech synthesis canbe used to provide a dialog with the person, so that a display may notbe needed for menu presentation. In this case, menus can be presentedusing synthesized speech. For example, a person speaks the word “Diet”to the portable computing device. The device responds “Diet LogActivated. It is now 12:10 pm. Lunch food types include 1—burrito,2—sandwich, 3—yogurt, 4—other. Please speak number”. The person speaksthe number “1”. The device responds “Burrito chosen. Choose 1—bean,2—beef, 3—chicken.”. The person speaks the number “3”. The deviceresponds “Did you eat the whole chicken burrito”. The person responds“Yes”. The device responds “OK. Chicken burrito entered into diet log.Don't forget your run this afternoon.”

Interactive Television

A portable device can also be used for remote control of a digitalinteractive television or other entertainment device. Spoken commandscan be transmitted to the set-top box of the digital interactivetelevision, for example by speaking to the portable device with wirelesstransmission of a digitized audio file to the set-top box. The set-topbox, or other device such as a remote server in communication with it,has a voice recognition system to convert the digitized audio file toelectronic data, such as text and/or numbers. The electronic data can beused for channel selection of the digital interactive television, orselection from menus such as food item menus displayed on the digitalinteractive television. Spoken commands can be transmitted from aportable device, possibly with the form factor of a remote control, overa wireless telephone network to a remote computer having a speechrecognition system, which then responds to the received and interpretedcommands by providing corresponding visual menu presentations, selectionindications, channel selections, program selections, and the like on thedisplay of the interactive television.

Physical Location Determination

The portable computing device, or add-on hardware module, can furthercomprise a physical location determining unit, such as a globalpositioning system, triangulation system, or the like. The location canbe correlated with positions of food retail establishments using a foodretail database, and so with menus and available food lists. Hence, aportable computing device according to the present invention can presenta food menu to a person based on the person's location. If the person isin a restaurant, a menu can be presented on a display of the portablecomputing device. The person can then select from the presented menuusing the speech recognition systems described herein. A nutritionaldatabase is then used to assist in creating a diet log from the selecteditems.

Wrist-Mounted Device

Embodiments of the present invention can comprise a portable devicesupported by a strap around a wrist of the person, in the form of awristwatch. The display of the device can be used to display time, otherdata, entertainment, and food menus to the person. A microphone in thehousing can be used to receive spoken commands. A speech signal, such asa digitized audio file, can be transmitted to other devices carried bythe person, or with which the person interacts (for example a PDA,desktop computer, entertainment device, remote computer, and the like).The display of menu options can be presented on the display of anotherdevice, such as a PDA.

Limited Display Capabilities

In some cases, display complexity will be insufficient to display alloptions to the person in the form of a menu. In this case, the personcan speak the first letter of the required option, so as to limit thedisplay to those food items or food types beginning with that letter.Other letters or numbers can then be spoken so as to further limit thedisplay, or to select an item.

Other variations on the disclosed embodiments of the present inventionwill be apparent to those of skill in the art. These variations do notdepart from either the spirit or scope of the present invention, andtherefore the disclosed embodiment should be interpreted broadly.

1. A diet logging module for a portable computing device comprising: amodule housing configured to engage with an accessory interface of theportable computing device; a memory storing a database of nutritionaldata relating to each of a plurality of food item identifiers, each fooditem identifier comprising an alphanumeric character; a display, adaptedto display at least one menu of a set of hierarchical menus of objectsrelated to consumption of a food item of the plurality of food items,each object of the menu of objects identified by a corresponding objectidentifier, each object identifier comprising an alphanumeric character;a speech receiving device, adapted to receive a speech command from auser, the speech command comprising an alphanumeric commandcorresponding to an object identifier or a food item identifier; aspeech recognition system, adapted to receive the speech command fromthe speech receiving device, and to display a subsequent menu of the setof hierarchical menus in response to the speech command; a locationdetermination component, adapted to determine a location of the portabledevice in operation by the user; and a display process to display one ormore food items determined by a food item identifier received by thespeech recognition system, and further determined by the location of theportable device as determined by the location determination component.2. The diet logging module of claim 1, wherein the locationdetermination component is selected from the group consisting of aglobal positioning system, a cell phone triangulation system, and awireless network based location system, and further comprising acorrelation module that uses a food retail database to provide one ormore lists of food items from one or more food retail establishments tothe display process based on the location of the user.
 3. The dietlogging module of claim 2, wherein the display process displays a listof menu items to the user from the one or more lists of food items basedon the location of the portable device, and further comprising a timedetermination means, and wherein the display process displays at leastcertain food items from the one or more lists of food items based on thetime of operation by the user.
 4. The diet logging module of claim 3,wherein the software application program is adapted to select a selectedfood item identifier from the nutritional database in response to aspeech command received for a food selection menu of the hierarchicalset of menus, the food selection menu including a plurality of differentfood items corresponding to parameters specified in previously displayedmenus of the hierarchical set of menus.
 5. The diet logging module ofclaim 4, wherein the software application program is further adapted tostore the selected food item identifier and associated nutritional datain the memory so as to create a diet log generated by the diet logprocess.
 6. A portable computing system comprising: a display; amicroprocessor; a memory; a database of nutritional data relating toeach of a plurality of food item identifiers, each food item identifiercomprising an alphanumeric character corresponding to a unique fooditem; a speech recognition system providing an electrical speech commandsignal correlated with a spoken input; a location determinationcomponent, adapted to determine a location of the portable device inoperation by a user; and a software application program, executed by themicroprocessor, adapted to display a set of hierarchical selectionmenus, each menu of the set of hierarchical selection menus comprisingan alphanumeric object identifier corresponding to a selection object,each selection object causing display of a subsequent menu of the set ofhierarchical menus until a food menu is displayed, the food menucomprising selections obtained from a food retail establishment based onthe location of the portable device, and wherein the food menu displaysfood item identifiers allowing a user to select a food item by utteringa speech command consisting of the alphanumeric food identifiercorresponding to a selected food item, the software application programfurther configured to store nutritional data relating to the selectedfood item identifier in the memory so as to create a diet log ofselected food items.
 7. The portable computing device of claim 6,wherein the location determination component is selected from the groupconsisting of a global positioning system, a cell phone triangulationsystem, and a wireless network based location system.
 8. The portablecomputing device of claim 6, wherein the set of hierarchical selectionmenus is selected from the group consisting of: meal type, food type,and portion size.
 9. An accessory module for a portable computingdevice, the portable computing device comprising a microprocessor, adisplay, and an accessory port, the accessory module comprising: anelectrical interface, adapted to communicate with the accessory port; amemory; a location determination component, adapted to determine alocation of the portable device in operation by a user; a nutritionaldatabase, stored in the memory, correlating each of a plurality of fooditem identifiers with an associated food item nutritional content, eachfood item identifier of the plurality of food item identifierscomprising an alphanumeric character corresponding to a unique fooditem; and a software application program, stored in the memory, andexecutable by the microprocessor and adapted to display a set ofhierarchical selection menus, each menu of the set of hierarchicalselection menus comprising an alphanumeric object identifiercorresponding to a selection object, each selection object causingdisplay of a subsequent menu of the set of hierarchical menus until afood menu is displayed, the food menu comprising selections based on thelocation of the portable device; and wherein the food menu displays fooditem identifiers allowing a user to select a food item by uttering aspeech command consisting of the alphanumeric food identifiercorresponding to a selected food item, the software application programfurther configured, and further adapted to store the food item selectionand correlated nutritional content within a diet log for the person. 10.The accessory module of claim 9, wherein the location determinationcomponent is selected from the group consisting of a global positioningsystem, a cell phone triangulation system, and a wireless network basedlocation system.
 11. The accessory module of claim 10, wherein softwareapplication program displays a list of menu items to the user based onthe location of the portable device, and further comprising a timedetermination means, and wherein the display process to displays the oneor more further corresponding to the time of operation by the user. 12.The accessory module of claim 9, wherein the speech command signal isprovided by one of a speech recognition system resident on the accessorymodule, and a speech recognition system resident on the portablecomputing device.
 13. A method of creating a diet log for a person,comprising: presenting a series of hierarchical food selection menus tothe person, each food selection menu comprising an alphanumericidentifier corresponding to a selection related to a food itemselectable by the person; receiving a type selection speech command fromthe person, the speech command comprising a spoken alphanumericcharacter for a menu item selected by the user; determining a locationof the person; determining a time of presenting the series ofhierarchical food selection menus to the person; displaying a subsequentfood selection menu in response to the speech command, the subsequentfood selection menu providing additional data relating to the selectablefood item, wherein the selectable food item depends of the location ofthe person, and the determined time; presenting a menu of food itemidentifiers to the person in a final food selection menu of the seriesof hierarchical food selection menus; receiving a food selection speechcommand from the person, the food selection speech command comprising aspoken alphanumeric character for a selected food item; and storing theselected food item identifier and associated nutritional data within amemory device, so as to create a diet log for the person.
 14. The methodof claim 13, further comprising obtaining one or more lists of fooditems from one or more food retail establishments based on the locationof the person, and wherein the selectable food item is selected from theone or more lists of food items.
 15. The method of claim 13, wherein themenus are presented to the person on the display of a portable computingdevice carried by the person.
 16. The method of claim 15, wherein theportable computing device comprises a wrist-mounted device worn by theperson.
 17. The method of claim 15, wherein the correlation of speechcommands with selected identifiers is performed by a softwareapplication program running on the portable computing device.
 18. Themethod of claim 15, wherein the correlation of speech commands withselected identifiers is performed by a software application programrunning on a remote computer system in communication with the portablecomputing device.